Electronic device for video editing with dynamic tone metadata and operation method thereof

ABSTRACT

An apparatus and method for editing an image including dynamic tone metadata in an electronic device are provided. The electronic device includes a display, and at least one processor operatively connected to the display, wherein the at least one processor may be configured to generate a third image to be inserted between a first image and a second image continuous with the first image among a plurality of images belonging to video content, generate dynamic tone metadata of the third image based on dynamic tone metadata of the first image and the second image, and update the video content by adding the third image and the dynamic tone metadata of the third image.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. §119(a) of a Korean patent application number 10-2019-0176520, filed onDec. 27, 2019, in the Korean Intellectual Property Office, thedisclosure of which is incorporated by reference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to an apparatus and method for editing a videoincluding dynamic tone metadata in an electronic device.

2. Description of Related Art

The dynamic range (DR) of an image may be defined as a ratio between thebrightest pixel value and the darkest pixel value in the image. Thehuman eye can perceive a relatively higher dynamic range than the camerasensor or display of a regular electronic device. Such an electronicdevice may be unable to obtain or represent an image of a dynamic rangeperceived by a human as it is. For example, when capturing an image of ascene in which a high-illuminance region (very bright region) and alow-illuminance region (very dark region) coexist, such as backlightphotography, the electronic device may obtain an image in whichinformation on the high-illuminance region or the low-illuminance regionis omitted. In addition, the image obtained by the electronic devicefails to represent colors or tones (or, brightness) perceived by humans.

To provide images of a quality required by the user, the electronicdevice may support a dynamic tone mapping technique capable ofrepresenting an image with a feeling similar to that of an imageperceived by a human. For example, a dynamic tone mapping technique maybe supported in high dynamic range (HDR) 10+ or Dolby Vision.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

A dynamic tone mapping technique may independently apply a tone mappingoperation to each frame (or image) constituting video content, so thatthe user can relatively accurately recognize the production intention ofthe video content. The dynamic tone mapping technique may provide tonemapping information in the form of metadata (dynamic tone metadata) tonaturally represent the tone of the video content created at abrightness of the mastering display (or monitor) (e.g., about 1000 to10000 nit) on the display of an electronic device with a differentbrightness (e.g., about 300 to 500 nit) according to the intention ofthe video producer.

The electronic device may edit video content based on a video editingtechnique such as frame rate conversion (FRC) or video concatenation.For example, when applying an FRC editing technique, to correspond tothe target frame rate, the electronic device may generate at least onenew frame (or image) and insert it between frame n (or image n) andframe n+1 (or image n+1) of the video content.

However, the electronic device may not have a scheme for generatingdynamic tone metadata corresponding to the new frame (or image)generated based on the FRC editing technique. When dynamic tone metadatacorresponding to at least some frames (or images) of the video contentto be reproduced does not exist, HDR processing corresponding to displaycharacteristics (e.g., brightness) of the electronic device may belimited. In this case, when the electronic device reproduces the videocontent, the image quality may deteriorate or a sense of heterogeneitymay occur.

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providean apparatus and method for editing a video including dynamic tonemetadata in an electronic device.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, an electronic device isprovided. The electronic device includes a display device, and at leastone processor operatively connected to the display device, wherein theat least one processor may be configured to generate a third image to beinserted between a first image and a second image continuous with thefirst image among a plurality of images belonging to video content,based on at least one of the first image or the second image, generatedynamic tone metadata of the third image based on dynamic tone metadataof the first image and dynamic tone metadata of the second image, andupdate the video content by adding the third image and the dynamic tonemetadata of the third image.

In accordance with another aspect of the disclosure, an electronicdevice is provided. The electronic device includes a display, and atleast one processor operatively connected to the display, wherein the atleast one processor may be configured to generate first video content byextracting at least one image from one or more video contents, identify,in case that there is a first image not having dynamic tone metadataamong at least one image included in the first video content, originalvideo content including the first image among the one or more videocontents, identify at least one image having dynamic tone metadata amonga plurality of images included in the original video content, select animage that satisfies a specified condition in playback time from amongthe identified at least one image having dynamic tone metadata, andgenerate dynamic tone metadata of the first image based on dynamic tonemetadata of the selected image.

In accordance with another aspect of the disclosure, an electronicdevice is provided. The electronic device includes a memory to store aplurality of images having dynamic tone metadata, a display, and atleast one processor operatively connected to the memory and the display,wherein the at least one processor may be configured to update a firstimage among a plurality of images belonging to video content based onvideo editing, select a second image corresponding to the updated firstimage from among the plurality of images stored in the memory or aplurality of images stored in another electronic device, and set dynamictone metadata of the updated first image by updating dynamic tonemetadata of the updated first image based on dynamic tone metadata ofthe second image corresponding to the updated first image.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription, taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a block diagram of an electronic device in a networkenvironment according to an embodiment of the disclosure;

FIG. 2 is a block diagram of an electronic device for editing dynamictone metadata according to an embodiment of the disclosure;

FIG. 3 is a flowchart for an electronic device to generate dynamic tonemetadata for an added image according to an embodiment of thedisclosure;

FIG. 4 illustrates generation of an additional image in an electronicdevice according to an embodiment of the disclosure;

FIG. 5 illustrates generation of an additional image based on atone-mapped image in an electronic device according to an embodiment ofthe disclosure;

FIG. 6 illustrates generation of an additional image based on atone-mapped image and dynamic tone metadata in an electronic deviceaccording to an embodiment of the disclosure;

FIGS. 7A and 7B illustrate generation of dynamic tone metadata in anelectronic device according to various embodiments of the disclosure;

FIG. 8 is a flowchart for an electronic device to generate dynamic tonemetadata according to an embodiment of the disclosure;

FIG. 9 illustrates generation of dynamic tone metadata based on a schemefor generating an additional image in an electronic device according toan embodiment of the disclosure;

FIGS. 10A and 10B illustrate generation of dynamic tone metadata basedon a reference ratio between reference images in an electronic deviceaccording to various embodiments of the disclosure;

FIG. 11 illustrates generation of dynamic tone metadata based on adistance to reference images in an electronic device according to anembodiment of the disclosure;

FIG. 12 is a flowchart for an electronic device to generate dynamic tonemetadata of an added image based on predefined dynamic tone metadataaccording to an embodiment of the disclosure;

FIG. 13 is a flowchart for an electronic device to edit dynamic tonemetadata of an image to which FRC editing is applied according to anembodiment of the disclosure;

FIG. 14 is a configuration for removing at least one image through FRCediting in an electronic device according to an embodiment of thedisclosure;

FIG. 15A is a flowchart for an electronic device to generate and editdynamic tone metadata of video content to which a video concatenationtechnique is applied according to an embodiment of the disclosure;

FIG. 15B is a flowchart for an electronic device to edit dynamic tonemetadata of video content to which a video concatenation technique isapplied according to an embodiment of the disclosure;

FIG. 16 is a configuration for generating video content through a videoconcatenation technique in an electronic device according to anembodiment of the disclosure;

FIG. 17 is a flowchart for an electronic device to edit dynamic tonemetadata of video content to which clip editing is applied according toan embodiment of the disclosure;

FIG. 18 is a configuration for generating video content through clipediting in an electronic device according to an embodiment of thedisclosure;

FIG. 19 is a flowchart for an electronic device to update dynamic tonemetadata of an edited image according to an embodiment of thedisclosure; and

FIGS. 20A to 20C illustrate encoding schemes of video content in anelectronic device according to various embodiments of the disclosure.

Throughout the drawings, like reference numerals will be understood torefer to like parts, components, and structures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

FIG. 1 is a block diagram illustrating an electronic device in a networkenvironment according to an embodiment of the disclosure.

Referring to FIG. 1, the electronic device 101 in the networkenvironment 100 may communicate with an electronic device 102 via afirst network 198 (e.g., a short-range wireless communication network),or an electronic device 104 or a server 108 via a second network 199(e.g., a long-range wireless communication network). According to anembodiment, the electronic device 101 may communicate with theelectronic device 104 via the server 108. According to an embodiment,the electronic device 101 may include a processor 120, memory 130, aninput device 150, a sound output device 155, a display device 160, anaudio module 170, a sensor module 176, an interface 177, a haptic module179, a camera module 180, a power management module 188, a battery 189,a communication module 190, a subscriber identification module (SIM)196, or an antenna module 197. In some embodiments, at least one (e.g.,the display device 160 or the camera module 180) of the components maybe omitted from the electronic device 101, or one or more othercomponents may be added in the electronic device 101. In someembodiments, some of the components may be implemented as singleintegrated circuitry. For example, the sensor module 176 (e.g., afingerprint sensor, an iris sensor, or an illuminance sensor) may beimplemented as embedded in the display device 160 (e.g., a display).

The processor 120 may execute, for example, software (e.g., a program140) to control at least one other component (e.g., a hardware orsoftware component) of the electronic device 101 coupled with theprocessor 120, and may perform various data processing or computation.According to an example embodiment, as at least part of the dataprocessing or computation, the processor 120 may load a command or datareceived from another component (e.g., the sensor module 176 or thecommunication module 190) in volatile memory 132, process the command orthe data stored in the volatile memory 132, and store resulting data innon-volatile memory 134. According to an embodiment, the processor 120may include a main processor 121 (e.g., a central processing unit (CPU)or an application processor (AP)), and an auxiliary processor 123 (e.g.,a graphics processing unit (GPU), an image signal processor (ISP), asensor hub processor, or a communication processor (CP)) that isoperable independently from, or in conjunction with, the main processor121. Additionally or alternatively, the auxiliary processor 123 may beadapted to consume less power than the main processor 121, or to bespecific to a specified function. The auxiliary processor 123 may beimplemented as separate from, or as part of the main processor 121.

The auxiliary processor 123 may control at least some of functions orstates related to at least one component (e.g., the display device 160,the sensor module 176, or the communication module 190) among thecomponents of the electronic device 101, instead of the main processor121 while the main processor 121 is in an inactive (e.g., sleep) state,or together with the main processor 121 while the main processor 121 isin an active state (e.g., executing an application). According to anembodiment, the auxiliary processor 123 (e.g., an image signal processoror a communication processor) may be implemented as part of anothercomponent (e.g., the camera module 180 or the communication module 190)functionally related to the auxiliary processor 123.

The memory 130 may store various data used by at least one component(e.g., the processor 120 or the sensor module 176) of the electronicdevice 101. The various data may include, for example, software (e.g.,the program 140) and input data or output data for a command relatedthereto. The memory 130 may include the volatile memory 132 or thenon-volatile memory 134. Non-volatile memory may include internal memory136 and/or external memory 138.

The program 140 may be stored in the memory 130 as software, and mayinclude, for example, an operating system (OS) 142, middleware 144, oran application 146.

The input device 150 may receive a command or data to be used by othercomponent (e.g., the processor 120) of the electronic device 101, fromthe outside (e.g., a user) of the electronic device 101. The inputdevice 150 may include, for example, a microphone, a mouse, a keyboard,or a digital pen (e.g., stylus pen).

The sound output device 155 may output sound signals to the outside ofthe electronic device 101. The sound output device 155 may include, forexample, a speaker or a receiver. The speaker may be used for generalpurposes, such as playing multimedia or playing record, and the receivermay be used for an incoming calls. According to an embodiment, thereceiver may be implemented as separate from, or as part of the speaker.

The display device 160 may visually provide information to the outside(e.g., a user) of the electronic device 101. The display device 160 mayinclude, for example, a display, a hologram device, or a projector andcontrol circuitry to control a corresponding one of the display,hologram device, and projector. According to an embodiment, the displaydevice 160 may include touch circuitry adapted to detect a touch, orsensor circuitry (e.g., a pressure sensor) adapted to measure theintensity of force incurred by the touch.

The audio module 170 may convert a sound into an electrical signal andvice versa. According to an embodiment, the audio module 170 may obtainthe sound via the input device 150, or output the sound via the soundoutput device 155 or a headphone of an external electronic device (e.g.,an electronic device 102) directly (e.g., wiredly) or wirelessly coupledwith the electronic device 101.

The sensor module 176 may detect an operational state (e.g., power ortemperature) of the electronic device 101 or an environmental state(e.g., a state of a user) external to the electronic device 101, andthen generate an electrical signal or data value corresponding to thedetected state. According to an embodiment, the sensor module 176 mayinclude, for example, a gesture sensor, a gyro sensor, an atmosphericpressure sensor, a magnetic sensor, an acceleration sensor, a gripsensor, a proximity sensor, a color sensor, an infrared (IR) sensor, abiometric sensor, a temperature sensor, a humidity sensor, or anilluminance sensor.

The interface 177 may support one or more specified protocols to be usedfor the electronic device 101 to be coupled with the external electronicdevice (e.g., the electronic device 102) directly (e.g., wiredly) orwirelessly. According to an embodiment, the interface 177 may include,for example, a high definition multimedia interface (HDMI), a universalserial bus (USB) interface, a secure digital (SD) card interface, or anaudio interface.

A connecting terminal 178 may include a connector via which theelectronic device 101 may be physically connected with the externalelectronic device (e.g., the electronic device 102). According to anembodiment, the connecting terminal 178 may include, for example, a HDMIconnector, a USB connector, a SD card connector, or an audio connector(e.g., a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanicalstimulus (e.g., a vibration or a movement) or electrical stimulus whichmay be recognized by a user via his tactile sensation or kinestheticsensation. According to an embodiment, the haptic module 179 mayinclude, for example, a motor, a piezoelectric element, or an electricstimulator.

The camera module 180 may capture a still image or moving images.According to an embodiment, the camera module 180 may include one ormore lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to theelectronic device 101. According to an example embodiment, the powermanagement module 188 may be implemented as at least part of, forexample, a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of theelectronic device 101. According to an embodiment, the battery 189 mayinclude, for example, a primary cell which is not rechargeable, asecondary cell which is rechargeable, or a fuel cell.

The communication module 190 may support establishing a direct (e.g.,wired) communication channel or a wireless communication channel betweenthe electronic device 101 and the external electronic device (e.g., theelectronic device 102, the electronic device 104, or the server 108) andperforming communication via the established communication channel. Thecommunication module 190 may include one or more communicationprocessors that are operable independently from the processor 120 (e.g.,the application processor (AP)) and supports a direct (e.g., wired)communication or a wireless communication. According to an embodiment,the communication module 190 may include a wireless communication module192 (e.g., a cellular communication module, a short-range wirelesscommunication module, or a global navigation satellite system (GNSS)communication module) or a wired communication module 194 (e.g., a localarea network (LAN) communication module or a power line communication(PLC) module). A corresponding one of these communication modules maycommunicate with the external electronic device via the first network198 (e.g., a short-range communication network, such as Bluetooth™wireless-fidelity (Wi-Fi) direct, or infrared data association (IrDA))or the second network 199 (e.g., a long-range communication network,such as a cellular network, the Internet, or a computer network (e.g.,LAN or wide area network (WAN)). These various types of communicationmodules may be implemented as a single component (e.g., a single chip),or may be implemented as multi components (e.g., multi chips) separatefrom each other. The wireless communication module 192 may identify andauthenticate the electronic device 101 in a communication network, suchas the first network 198 or the second network 199, using subscriberinformation (e.g., international mobile subscriber identity (IMSI))stored in the subscriber identification module 196.

The antenna module 197 may transmit or receive a signal or power to orfrom the outside (e.g., the external electronic device) of theelectronic device 101. According to an embodiment, the antenna module197 may include an antenna including a radiating element composed of aconductive material or a conductive pattern formed in or on a substrate(e.g., printed circuit board (PCB)). According to an embodiment, theantenna module 197 may include a plurality of antennas. In such a case,at least one antenna appropriate for a communication scheme used in thecommunication network, such as the first network 198 or the secondnetwork 199, may be selected, for example, by the communication module190 (e.g., the wireless communication module 192) from the plurality ofantennas. The signal or the power may then be transmitted or receivedbetween the communication module 190 and the external electronic devicevia the selected at least one antenna. According to an embodiment,another component (e.g., a radio frequency integrated circuit (RFIC))other than the radiating element may be additionally formed as part ofthe antenna module 197.

At least some of the above-described components may be coupled mutuallyand communicate signals (e.g., commands or data) therebetween via aninter-peripheral communication scheme (e.g., a bus, general purposeinput and output (GPIO), serial peripheral interface (SPI), or mobileindustry processor interface (MIPI)).

According to an embodiment, commands or data may be transmitted orreceived between the electronic device 101 and the external electronicdevice 104 via the server 108 coupled with the second network 199. Eachof the external electronic devices 102 and 104 may be a device of a sametype as, or a different type, from the electronic device 101. Accordingto an embodiment, all or some of operations to be executed at theelectronic device 101 may be executed at one or more of the externalelectronic devices 102, 104, or 108. For example, if the electronicdevice 101 should perform a function or a service automatically, or inresponse to a request from a user or another device, the electronicdevice 101, instead of, or in addition to, executing the function or theservice, may request the one or more external electronic devices toperform at least part of the function or the service. The one or moreexternal electronic devices receiving the request may perform the atleast part of the function or the service requested, or an additionalfunction or an additional service related to the request, and transferan outcome of the performing to the electronic device 101. Theelectronic device 101 may provide the outcome, with or without furtherprocessing of the outcome, as at least part of a reply to the request.To that end, a cloud computing, distributed computing, or client-servercomputing technology may be used, for example.

FIG. 2 is a block diagram of an electronic device for editing dynamictone metadata according to an embodiment of the disclosure. For example,an electronic device 201 may correspond to the electronic device 101 inFIG. 1.

Referring to FIG. 2, a processor 210 of the electronic device 201 mayinclude a decoding module 212, a video editing module 214, a metadataediting module 216, and an encoding module 218. In one embodiment, theprocessor 210 may be substantially the same as the processor 120 in FIG.1 or may be included in the processor 120. For example, the processor210 may include an application processor (AP), an image signal processor(ISP), or a display drive integrated circuit (DDI). For example,individual modules 212, 214, 216 and/or 218 included in the processor210 may be implemented in software, and the processor 210 may loadrelated instructions from the memory (e.g., memory 130 in FIG. 1) forexecution.

In various embodiments, the decoding module 212 may decode encoded videocontent stored in the memory (e.g., memory 130 in FIG. 1). In oneembodiment, the decoding module 212 may decode video content selectedfor editing among a plurality of encoded video content stored in thememory. For example, the video content may include a plurality of imagesconsecutive in time. For example, an image included in the video contentmay correspond to a frame constituting the video content.

In various embodiments, the video editing module 214 may generate oredit at least one image related to the video content decoded by thedecoding module 212. For example, video editing may include at least oneof frame rate conversion (FRC), video concatenation, clipping, scaling(e.g., image enlargement or reduction), object (e.g., graphic data ortext) insertion, or filtering.

In one embodiment, the video editing module 214 may generate at leastone new image and insert it between images so as to correspond to atarget frame rate based on an FRC editing technique (e.g., framerate upconversion). For example, a new image may be generated throughinterpolation based on at least one adjacent image included in the videocontent.

In one embodiment, the video editing module 214 may skip at least oneimage from consecutive images included in the video content so as tocorrespond to a target frame rate based on an FRC editing technique(e.g., framerate down conversion).

In one embodiment, the video editing module 214 may generate third videocontent by concatenating at least a portion of first video content andat least a portion of second video content based on a videoconcatenation editing technique.

In one embodiment, the video editing module 214 may adjust the size ofat least one image included in the video content based on a scalingediting technique. For example, the video editing module 214 mayincrease or decrease the size of at least one image as a whole or atleast some thereof.

In one embodiment, the video editing module 214 may insert graphic ortext data into at least a region of at least one image included in thevideo content based on an object insertion editing technique.

In one embodiment, the video editing module 214 may generate fourthvideo content by extracting at least a portion of first video contentbased on a clip editing technique.

In various embodiments, the metadata editing module 216 may generate oredit dynamic tone metadata corresponding to at least one image based onthe video content edited by the video editing module 214. For example,dynamic tone metadata may include information related to a tone (orbrightness) for representing an image.

In one embodiment, when at least one image is created based on an FRCediting technique (e.g., framerate up conversion), the metadata editingmodule 216 may generate dynamic tone metadata of the new image based ondynamic tone metadata of at least one reference image. For example, themetadata editing module 216 may generate dynamic tone metadata of a newimage based on an average of dynamic tone metadata of a plurality ofreference images related to the new image. As another example, themetadata editing module 216 may generate dynamic tone metadata of a newimage by applying a weight to dynamic tone metadata of the plurality ofreference images related to the new image. Here, the weight may bedetermined based on at least one of a time interval between the newimage and the reference image, a reference ratio between referenceimages related to generation of the new image, or a reference ratiobetween reference images related to encoding of the new image. Asanother example, the metadata editing module 216 may use dynamic tonemetadata of an image matching the new image, among at least one imageincluding dynamic tone metadata stored in the memory (e.g., memory 130in FIG. 1) or the server (e.g., server 108 in FIG. 1), as dynamic tonemetadata of the new image. Here, the at least one image includingdynamic tone metadata stored in the memory or the server may include atleast one image previously generated (or obtained) by the electronicdevice 201 and/or at least one image generated by an expert. As anotherexample, the metadata editing module 216 may generate dynamic tonemetadata of a new image by copying dynamic tone metadata of a referenceimage related to the new image from the video editing module 214.

In one embodiment, when at least one image is omitted from images by anFRC editing technique (e.g., framerate down conversion), the metadataediting module 216 may edit (or update) the dynamic tone metadata of theremaining images. For example, if the difference (or difference value)in dynamic tone metadata between images adjacent in playback time amongthe remaining images of the video content is greater than a threshold,the metadata editing module 216 may determine to edit the dynamic tonemetadata of images. The metadata editing module 216 may update thedynamic tone metadata of the current image based on the dynamic tonemetadata of the current image and/or at least one adjacent image. Forexample, the difference in dynamic tone metadata may include adifference between tone mapping curves representing tones (orbrightness) corresponding to individual images.

In one embodiment, when third video content is generated based on avideo concatenation editing technique, the metadata editing module 216may generate or edit dynamic tone metadata of at least one imageincluded in the third video content. For example, when there is nodynamic tone metadata associated with a first image included in thethird video content, the metadata editing module 216 may generatedynamic tone metadata of the first image based on dynamic tone metadataof a second image included in the original video content (e.g., firstvideo content or second video content). For instance, the dynamic tonemetadata of the first image may be generated by copying the dynamic tonemetadata of the second image. For instance, the first image maycorrespond to any image that does not have dynamic tone metadata in thethird video content. For instance, the first image may correspond to animage at the start position of each original video content (e.g., firstvideo content and/or second video content) within the third videocontent. For instance, the second image may correspond to an imageclosest to the first image in playback time among one or more images ofthe original video content having dynamic tone metadata and precedingthe first image in playback time. For instance, the second image maycorrespond to an image closest to the first image in playback time amongat least one image having dynamic tone metadata in the original videocontent. For example, if the difference in dynamic tone metadata betweenimages at a time point when different pieces of video content areconcatenated (e.g., boundary time point) exceeds a threshold value, themetadata editing module 216 may determine to edit the dynamic tonemetadata of images. The metadata editing module 216 may update dynamictone metadata of the current image based on dynamic tone metadata of atleast one of the current image, at least one previous image, or at leastone subsequent image. For instance, the image related to the boundarytime point may correspond to at least one image disposed at a positionwhere the first video content and the second video content areconcatenated within the third video content generated based on a videoconcatenation editing technique.

In one embodiment, when at least one image included in the video contentis edited based on a video editing technique (e.g., image enlargement,image reduction, or object insertion), the metadata editing module 216may edit (or update) the dynamic tone metadata of the edited image. Forexample, the metadata editing module 216 may edit dynamic tone metadataof at least one image based on a video editing technique correspondingto editing of at least one image. As another example, the metadataediting module 216 may use dynamic tone metadata of an image matchingthe edited image, among at least one image including dynamic tonemetadata and stored in the memory (e.g., memory 130 in FIG. 1) or theserver (e.g., server 108 in FIG. 1), as dynamic tone metadata of theedited image. For instance, editing (or updating) the dynamic tonemetadata of an edited image may be performed when the difference indynamic tone metadata between the edited image in the video content andan image adjacent to the edited image in playback time exceeds athreshold value.

In one embodiment, when fourth video content is generated by a clipediting technique, the metadata editing module 216 may generate dynamictone metadata of at least one image included in the fourth videocontent. For example, when there is no dynamic tone metadata for a firstimage included in the fourth video content, the metadata editing module216 may generate dynamic tone metadata of the first image by copying thedynamic tone metadata of a second image included in the original videocontent (e.g., first video content). For instance, the second image maycorrespond to an image closest to the first image in playback time amongone or more images of the original video content having dynamic tonemetadata and preceding the first image in playback time. For instance,the second image may correspond to an image closest to the first imagein playback time among at least one image having dynamic tone metadatain the original video content.

In various embodiments, the encoding module 218 may match image dataedited (or generated) by the video editing module 214 with dynamic tonemetadata edited (or generated) by the metadata editing module 216 andencode it in a predefined encoding scheme. In one embodiment, theencoding module 218 may re-encode edited video content by adding a newimage and dynamic tone metadata to the bit stream of the original imageof the video content. In one embodiment, the encoding module 218 mayencode the new image and dynamic tone metadata separately from (orindependently of) the original image and insert it into the bit streamof the original image. For example, the new image and dynamic tonemetadata may be encoded as an intra picture (e.g., I picture). Asanother example, the new image and dynamic tone metadata may be encodedas a P picture referring to the reference (e.g., I picture) of theoriginal image. For instance, the predefined encoding scheme mayinclude, but not limited to, high efficiency video coding (HEVC) orH.264 encoding scheme.

According to various embodiments, the electronic device (e.g.,electronic device 101 in FIG. 1 or electronic device 201 in FIG. 2) mayinclude: a display device (e.g., display device 160 in FIG. 1); and atleast one processor (e.g., processor 120 in FIG. 1) operativelyconnected to the display device, wherein the at least one processor maybe configured to generate a third image to be inserted between a firstimage and a second image continuous with the first image among aplurality of images included in video content, generate dynamic tonemetadata of the third image based on dynamic tone metadata of the firstimage and the second image, and update the video content by adding thethird image and the dynamic tone metadata of the third image.

In various embodiments, the at least one processor may be configured togenerate dynamic tone metadata of the third image based on an average ofdynamic tone metadata of the first image and dynamic tone metadata ofthe second image.

In various embodiments, the at least one processor may be configured to:identify a weight corresponding to the first image and the second imagebased on at least one of a reference ratio of the first image and thesecond image referenced to generate the third image, a time intervalbetween the first image, the second image, and the third image, asequence including the first image, the second image, and the thirdimage, or group of pictures (GOP) information including the first image,the second image, and the third image; and apply the weight to dynamictone metadata of the first image and dynamic tone metadata of the secondimage to generate dynamic tone metadata of the third image. For example,the image sequence and/or GOP may be set based on a scene change.

In various embodiments, the electronic device may further include amemory (e.g., memory 130 in FIG. 1) to store a plurality of imageshaving dynamic tone metadata. The at least one processor may beconfigured to select an image corresponding to the third image fromamong the plurality of images stored in the memory or a plurality ofimages stored in an external electronic device, and set dynamic tonemetadata of the third image to dynamic tone metadata of the selectedimage corresponding to the third image.

In various embodiments, the at least one processor may be configured togenerate the third image based on the second image when a differencebetween the first image and the second image satisfies a specifiedcondition, and generate the third image based on the first image and thesecond image when the difference between the first image and the secondimage does not satisfy the specified condition.

In various embodiments, when the third image is generated based on thesecond image, the at least one processor may be configured to setdynamic tone metadata of the third image to dynamic tone metadata of thesecond image.

According to various embodiments, the electronic device (e.g.,electronic device 101 in FIG. 1 or electronic device 201 in FIG. 2) mayinclude: a display device (e.g., display device 160 in FIG. 1); and atleast one processor (e.g., processor 120 in FIG. 1) operativelyconnected to the display device, wherein the at least one processor maybe configured to: generate first video content by extracting at leastone image from one or more of video contents; identify, when there is afirst image not having dynamic tone metadata among at least one imageincluded in the first video content, the original video contentincluding the first image among the one or more of video contents;identify at least one image having dynamic tone metadata among aplurality of images included in the original video content; select animage that satisfies a specified condition in playback time from amongthe identified at least one image having dynamic tone metadata; andgenerate dynamic tone metadata of the first image based on dynamic tonemetadata of the selected image.

In various embodiments, the at least one processor may be configured toidentify at least one image preceding the first image in playback timeamong one or more images of the original video content having dynamictone metadata, and select an image closest to the first image inplayback time among the at least one image preceding the first image inplayback time.

In various embodiments, the at least one processor may be configured toselect an image closest to the first image in playback time among atleast one image of the original video content having dynamic tonemetadata.

In various embodiments, the at least one processor may be configured togenerate dynamic tone metadata of the first image by copying dynamictone metadata of the selected image.

In various embodiments, the at least one processor may be configured togenerate the first video content by extracting at least one image fromsecond video content.

In various embodiments, the at least one processor may be configured togenerate the first video content by connecting at least one imageincluded in second video content and at least one image included inthird video content.

In various embodiments, the at least one processor may be configured tocompare dynamic tone metadata of a second image being the last of atleast one image of the second video content included in the first videocontent with dynamic tone metadata of a third image being a first of atleast one image of the third video content included in the first videocontent, wherein the third image is continuous with the second imagewithin the first video content, and update the dynamic tone metadata ofthe third image based on the dynamic tone metadata of the second imageif the difference in dynamic tone metadata between the second image andthe third image satisfies a specified condition.

In various embodiments, if the difference in dynamic tone metadatabetween the second image and the third image satisfies the specifiedcondition, the at least one processor may be configured to update thedynamic tone metadata of the third image based on the dynamic tonemetadata of at least one of the second image, the third image, or afourth image continuous with the third image.

In various embodiments, the at least one processor may be configured toupdate the dynamic tone metadata of the third image based on an averageof dynamic tone metadata of at least two of the second image, the thirdimage, or the fourth image.

In various embodiments, the at least one processor may be configured toidentify a weight corresponding to the second image, the third image,and the fourth image based on at least one of a sequence including thesecond image, the third image and the fourth image, or information on agroup of pictures (GOP) including the second image, the third image andthe fourth image, and update dynamic tone metadata of the third image byapplying the weight to dynamic tone metadata of at least one of thesecond image, the third image, or the fourth image.

In various embodiments, the at least one processor may be configured tomaintain dynamic tone metadata of the third image if the difference indynamic tone metadata between the second image and the third image doesnot satisfy the specified condition.

According to various embodiments, the electronic device (e.g.,electronic device 101 in FIG. 1 or electronic device 201 in FIG. 2) mayinclude: a memory (e.g., memory 130 in FIG. 1) to store a plurality ofimages having dynamic tone metadata; a display (e.g., display device 160in FIG. 1); and at least one processor (e.g., processor 120 in FIG. 1)operatively connected to the memory and the display, wherein the atleast one processor may be configured to: update a first image among aplurality of images belonging to video content based on video editing;select a second image corresponding to the updated first image fromamong the plurality of images stored in the memory or a plurality ofimages stored in another electronic device; and set dynamic tonemetadata of the updated first image by updating dynamic tone metadata ofthe first image based on dynamic tone metadata of the second imagecorresponding to the updated first image.

In various embodiments, the at least one processor may be configured toset dynamic tone metadata of the updated first image by copying dynamictone metadata of the second image corresponding to the updated firstimage.

In various embodiments, the video editing may include at least one ofscaling, object insertion, or filtering.

FIG. 3 is a flowchart for an electronic device to generate dynamic tonemetadata for an added image according to an embodiment of thedisclosure.

In the following embodiments, operations may be performed in sequencebut are not necessarily performed in sequence. For example, operationsmay be performed differently from the listed order, and at least twooperations may be performed in parallel. For instance, the electronicdevice may be the electronic device 101 in FIG. 1 or the electronicdevice 201 in FIG. 2. In the following description, at least someoperations of the flowchart 300 of FIG. 3 will be described withreference to FIGS. 4, 5, 6, 7A, and 7B.

FIG. 4 illustrates generation of an additional image in an electronicdevice according to an embodiment of the disclosure.

FIG. 5 illustrates generation of an additional image based on atone-mapped image in an electronic device according to an embodiment ofthe disclosure.

FIG. 6 illustrates generation of an additional image based on atone-mapped image and dynamic tone metadata in an electronic deviceaccording to an embodiment of the disclosure.

FIGS. 7A and 7B illustrate generation of dynamic tone metadata in anelectronic device according to various embodiments of the disclosure.

Referring to FIG. 3, in various embodiments, at operation 301, theelectronic device (e.g., processor 120 in FIG. 1 or processor 210 inFIG. 2) may detect an occurrence of a video editing event. In oneembodiment, when selection of an edit menu for video content is detectedaccording to a user input, the processor 210 may determine that a videoediting event corresponding to the edit menu has occurred.

In various embodiments, at operation 303, the electronic device (e.g.,processor 120 or 210) may generate an additional image to be insertedinto the video content through video editing corresponding to the videoediting event. In one embodiment, when performing FRC editing (e.g.,framerate up conversion) based on the video editing event, the processor210 may generate at least one new image to be inserted between images ofthe video content so as to correspond to the target frame rate.

Referring to FIG. 4, the processor 210 may generate image n+0.5 (420)through interpolation based on image n (400) and image n+1 (410).

In various embodiments, at operation 305, the electronic device (e.g.,processor 120 or 210) may generate dynamic tone metadata of theadditional image based on dynamic tone metadata of at least onereference image. In one embodiment, when image n+0.5 (420) is generatedas shown in FIG. 4, the processor 210 may generate dynamic tone metadata422 of image n+0.5 (420) based on dynamic tone metadata 402 of image n(400) and dynamic tone metadata 412 of image n+1 (410). For instance,the reference images of image n+0.5 (420) may include image n (400) andimage n+1 (410) having been referenced to generate image n+0.5 (420).

Referring to FIG. 7A, the processor 210 may generate dynamic tonemetadata 720 of image n+0.5 based on the average of dynamic tonemetadata 700 of image n and dynamic tone metadata 710 of image n+1.

Referring to FIG. 7B, the processor 210 may generate dynamic tonemetadata 750 of image n+0.5 based on the average of dynamic tonemetadata 730 of image n and dynamic tone metadata 740 of image n+1. Forinstance, dynamic tone metadata is information related to the tone (orbrightness) to represent an image according to the intention of anartist, and may include a tone mapping curve corresponding to the ratiobetween the target nit and the source nit as shown in FIG. 7B. Forinstance, the source nit may correspond to the brightness of themastering display (or monitor) at the time of production of the videocontent, and the target nit may correspond to the brightness of thedisplay of an electronic device (e.g., electronic device 201) forreproducing the video content. For instance, ‘n’ is identificationinformation of an image included in the video content, and may be aninteger value.

In various embodiments, at operation 307, the electronic device (e.g.,processor 120 or 210) may store the video content including theadditional image and dynamic tone metadata thereof. In one embodiment,the processor 210 may insert the additional image and dynamic tonemetadata thereof into the bit stream of the video content for storage.For example, the processor 210 may insert the additional image anddynamic tone metadata thereof into video content of a first FPS (e.g.,240 FPS) based on the FPS editing to generate video content of a secondFPS (e.g., 480 FPS or 960 FPS). In this case, when the video content ofthe second FPS is encoded at a reference FPS (e.g., 30 FPS), it may bereproduced as images where the motion of the object is slow (e.g., slowmotion) compared to the video content of the first FPS.

In various embodiments, the electronic device (e.g., processor 120 or210) may generate an additional image based on a tone-mapped image.

Referring to FIG. 5, the processor 210 may generate a motion vector(524) through motion estimation based on tone-mapped image n (504) andtone-mapped image n+1 (514). Image n+0.5 (520) may be generated throughmotion compensation using the motion vector (524) based on image n (500)(not tone-mapped) and image n+1 (510) (not tone-mapped). The dynamictone metadata 522 of image n+0.5 (520) may be generated based on dynamictone metadata 502 of image n (500) and dynamic tone metadata 512 ofimage n+1 (510). For instance, tone-mapped image n (504) may correspondto image n (500) mapped to the brightness of the display of theelectronic device 201 based on the dynamic tone metadata 502 of image n(500).

Referring to FIG. 6, the processor 210 may generate tone-mapped imagen+0.5 (624) through interpolation based on tone-mapped image n (604) andtone-mapped image n+1 (614). The processor 210 may generate dynamic tonemetadata 622 of image n+0.5 (620) based on dynamic tone metadata 602 ofimage n (600) and dynamic tone metadata 612 of image n+1 (610). Imagen+0.5 (620) may be generated through inverse tone mapping of tone-mappedimage n+0.5 (624) based on the dynamic tone metadata 622.

FIG. 8 is a flowchart for an electronic device to generate dynamic tonemetadata according to an embodiment of the disclosure.

The operations of FIG. 8 in flowchart 800 described below may be detailsof operation 303 and operation 305 in FIG. 3. In the followingembodiments, operations may be performed in sequence but are notnecessarily performed in sequence. For example, operations may beperformed differently from the listed order, and at least two operationsmay be performed in parallel. For instance, the electronic device may bethe electronic device 101 in FIG. 1 or the electronic device 201 in FIG.2. In the following description, at least some operations of FIG. 3 willbe described with reference to FIGS. 9, 10A, 10B, and 11.

FIG. 9 illustrates generation of dynamic tone metadata based on a schemefor generating an additional image in an electronic device according toan embodiment of the disclosure.

FIGS. 10A and 10B illustrate generation of dynamic tone metadata basedon a reference ratio between reference images in an electronic deviceaccording to various embodiments of the disclosure.

FIG. 11 illustrates generation of dynamic tone metadata based on thedistance to reference images in an electronic device according to anembodiment of the disclosure.

Referring to FIG. 8, in various embodiments, when the electronic device(e.g., processor 120 in FIG. 1 or processor 210 in FIG. 2) detects anoccurrence of an event for video editing (e.g., operation 301 in FIG.3), at operation 801, it may determine whether the difference betweenimages (e.g., reference images) between which the additional image isinserted exceeds a first threshold. In one embodiment, as shown in FIG.9, when generating an image to be inserted between image n (900) andimage n+1 (910), the processor 210 may calculate the difference betweenimage n (900) and image n+1 (910). For instance, the difference betweenimages may include a difference in pixel value between a first image anda second image or a difference in feature point therebetween. Forinstance, the first threshold may include a reference value fordetermining whether images are similar.

In various embodiments, if the difference between the images betweenwhich the additional image is inserted exceeds the first threshold(e.g., “yes” branch of operation 801), at operation 803, the electronicdevice (e.g., processor 120 or 210) may generate the additional imagethrough a first scheme (e.g., fallback). In one embodiment, when thedifference between images between which the additional image is insertedexceeds the first threshold, the processor 210 may determine that ascene change has occurred or the object movement has been relativelylarge.

Referring to FIG. 9, the processor 210 may generate image n+0.5 (920) bycopying image n+1 (910) according to the first scheme (e.g., fallback).

In various embodiments, at operation 805, the electronic device (e.g.,processor 120 or 210) may generate dynamic tone metadata of theadditional image by copying dynamic tone metadata of the image that isreferenced to generate the additional image according to the firstscheme (e.g., fallback). In one embodiment, as shown in FIG. 9, whenimage n+0.5 (920) is generated by copying image n+1 (910), the processor210 may generate dynamic tone metadata 922 of image n+0.5 (920) bycopying dynamic tone metadata 912 of image n+1 (910). For example, theprocessor 210 may generate dynamic tone metadata 922 of image n+0.5(920) by using dynamic tone metadata 912 of the image n+1 (910)regardless of dynamic tone metadata 902 of the image n 900.

In various embodiments, if the difference between the images betweenwhich the additional image is inserted is less than or equal to thefirst threshold (e.g., “no” branch of operation 801), at operation 807,the electronic device (e.g., processor 120 or 210) may generate theadditional image through a second scheme (e.g., interpolation). In oneembodiment, when the difference between images between which theadditional image is inserted is less than or equal to the firstthreshold, the processor 210 may determine that the images between whichthe additional image is inserted are relatively similar. In this case,as shown in FIG. 4, the processor 210 may generate image n+0.5 (420)through the second scheme (e.g., interpolation) based on image n (400)and image n+1 (410).

In various embodiments, at operation 809, the electronic device (e.g.,processor 120 or 210) may generate dynamic tone metadata of theadditional image based on dynamic tone metadata of the image that isreferenced to generate the additional image according to the secondscheme (e.g., interpolation). In one embodiment, the processor 210 maygenerate dynamic tone metadata of the additional image by applying aweight corresponding to a ratio (e.g., reference ratio) betweenreference images referenced to generate the additional image to dynamictone metadata of the reference images. For example, image n+0.5 may begenerated by referencing image n at about 80 percent and image n+1 atabout 20 percent.

Referring to FIG. 10A, dynamic tone metadata 1020 of image n+0.5 may begenerated by applying a first weighting coefficient (e.g., about 0.8) tothe dynamic tone metadata 1000 of image n and applying a secondweighting coefficient (e.g., about 0.2) to the dynamic tone metadata1010 of image n+1. As another example, image n+0.5 may be generated byreferencing image n at about 20 percent and image n+1 at about 80percent.

Referring to FIG. 10B, dynamic tone metadata 1050 of image n+0.5 may begenerated by applying a first weighting coefficient (e.g., about 0.2) tothe dynamic tone metadata 1030 of image n and applying a secondweighting coefficient (e.g., about 0.8) to the dynamic tone metadata1040 of image n+1.

In one embodiment, the processor 210 may generate dynamic tone metadataof the additional image by applying a weight corresponding to the timeinterval between the additional image and the reference image to dynamictone metadata of the reference image.

Referring to FIG. 11, image n+0.5 (1120) may be inserted between image n(1100) and image n+1 (1110). In this case, the processor 210 maygenerate dynamic tone metadata of image n+0.5 (1120) by applying a thirdweighting coefficient (e.g., about 0.5) equally to dynamic tone metadataof image n (1100) and dynamic tone metadata of image n+1 (1110). Asanother example, as shown in FIG. 11, image n+0.25 (1122) may beinserted at a position relatively closer to image n (1100) (e.g., about0.25) than image n+1 (1110). In this case, the processor 210 maygenerate dynamic tone metadata of image n+0.25 (1122) by applying afourth weighting coefficient (e.g., about 0.75) to the dynamic tonemetadata of image n (1100) and applying a fifth weighting coefficient(e.g., about 0.25) to the dynamic tone metadata of image n+1 (1110). Asanother example, as shown in FIG. 11, image n+0.75 (1124) may beinserted at a position relatively closer to image n+1 (1110) (e.g.,about 0.75) than image n (1100). In this case, the processor 210 maygenerate dynamic tone metadata of image n+0.75 (1124) by applying afifth weighting coefficient (e.g., about 0.25) to the dynamic tonemetadata of image n (1100) and applying a fourth weighting coefficient(e.g., about 0.75) to the dynamic tone metadata of image n+1 (1110).

In various embodiments, when encoding video content including anadditional image, the electronic device (e.g., processor 120 or 210) maygenerate dynamic tone metadata of the additional image by applying aweight corresponding to the ratio (e.g., reference ratio) betweenreference images referenced in encoding the additional image to thedynamic tone metadata of the reference images.

In various embodiments, the electronic device (e.g., processor 120 or210) may set a weighting coefficient to be applied to dynamic tonemetadata of a reference image in units of a sequence of scenes or inunits of a group of pictures (GOP) according to whether the additionalimage and the reference image belong to the same scene or GOP. In oneembodiment, when the additional image and the reference images belong tothe same sequence or GOP, the electronic device may set weightingcoefficients to be applied to the dynamic tone metadata of the referenceimages based on at least one of a reference ratio between the referenceimages or time intervals between the additional image and the referenceimages in order to generate dynamic tone metadata of the additionalimage. In one embodiment, when at least one of the reference imagesbelongs to a sequence or GOP different from that of the additionalimage, the electronic device may minimize or zero the weightingcoefficient to be applied to dynamic tone metadata of the referenceimage belonging to the different sequence or GOP.

FIG. 12 is a flowchart for an electronic device to generate dynamic tonemetadata of an added image based on predefined dynamic tone metadataaccording to an embodiment of the disclosure.

The operations in flowchart 1200 of FIG. 12 described below may bedetails of operation 305 in FIG. 3. In the following embodiments,operations may be performed in sequence but are not necessarilyperformed in sequence. For example, operations may be performeddifferently from the listed order, and at least two operations may beperformed in parallel. For instance, the electronic device may be theelectronic device 101 in FIG. 1 or the electronic device 201 in FIG. 2.

Referring to FIG. 12, when the additional image is generated based onvideo editing (e.g., operation 303 in FIG. 3), at operation 1201, theelectronic device (e.g., processor 120 in FIG. 1 or processor 210 inFIG. 2) may detect an image matching the additional image among imageshaving dynamic tone metadata stored in the memory (e.g., memory 130 inFIG. 1) or a server (e.g., server 108 in FIG. 1). In one embodiment, theprocessor 210 may select an image having the smallest difference fromthe additional image among the images stored in the memory or the serveras an image matching the additional image. For instance, the differencefrom the additional image may indicate a difference in pixel value orfeature point between the additional image and an image stored in thememory or server. For example, the images stored in the memory or theserver may include at least one image generated by the electronic device201 using a camera (e.g., camera module 180 in FIG. 1) and/or at leastone image generated by an expert. For example, the images stored in thememory or the server may be stored as image data itself, or may bestored in the form of at least one of sub-sampled data, differentialpulse code modulated (DPCM) image, feature point data, or frequencydomain data.

In various embodiments, at operation 1203, the electronic device (e.g.,processor 120 or 210) may identify dynamic tone metadata of the imagematching the additional image. For instance, the image matching theadditional image may indicate an image having a difference from theadditional image less than or equal to the first threshold.

In various embodiments, at operation 1205, the electronic device (e.g.,processor 120 or 210) may generate dynamic tone metadata of theadditional image based on dynamic tone metadata of the image matchingthe additional image. In one embodiment, the processor 210 may usedynamic tone metadata of the image matching the additional image asdynamic tone metadata of the additional image.

In various embodiments, the electronic device (e.g., processor 120 or210) may mark the additional image and its dynamic tone metadatainserted in the video content to be distinguished from the originalimages and their dynamic tone metadata. In one embodiment, the processor210 may configure the identification number of the additional imageseparately from the original image. In one embodiment, the processor 210may record the dynamic tone metadata twice in succession for theadditional image to indicate that the corresponding additional image isan image inserted through video editing. In one embodiment, theprocessor 210 may indicate that the additional image is an imageinserted through video editing by inserting specific information set asa dummy in the last part of the payload of the additional image. Forexample, when the additional image is encoded in joint photographyexperts group (JPEG), specific information set as a dummy foridentifying the additional image may be inserted behind the “end ofpicture” marker. As another example, when the additional image isencoded in high efficiency video coding (HEVC), specific information setas a dummy for identifying the newly added video may be inserted into“slice segment header extension data byte” at the end of slice data.Accordingly, when the electronic device 201 (or the user) edits (orre-edits) video content, it may perform editing or re-editing by usingonly the images desired by the user (e.g., original images), therebypreventing deterioration of the picture quality of the video content.

FIG. 13 is a flowchart for an electronic device to edit dynamic tonemetadata of an image to which FRC editing is applied according to anembodiment of the disclosure.

In the following embodiments, operations may be performed in sequencebut are not necessarily performed in sequence. For example, operationsmay be performed differently from the listed order, and at least twooperations may be performed in parallel. For instance, the electronicdevice may be the electronic device 101 in FIG. 1 or the electronicdevice 201 in FIG. 2. In the following description, at least someoperations of FIG. 13 in flowchart 1300 will be described with referenceto FIG. 14.

FIG. 14 is a configuration for removing at least one image through FRCediting in an electronic device according to an embodiment of thedisclosure.

Referring to FIG. 13, in various embodiments, at operation 1301, theelectronic device (e.g., processor 120 in FIG. 1 or processor 210 inFIG. 2) may omit at least one image of video content based on a videoediting technique (e.g., framerate down conversion).

Referring to FIG. 14, the processor 210 may omit at least some images1410 among consecutive images 1400, 1410 and 1420 to correspond to thetarget frame rate.

In various embodiments, at operation 1303, the electronic device (e.g.,processor 120 or 210) may check a difference in dynamic tone metadatabetween a plurality of images included in the video content edited bythe FRC editing technique. In one embodiment, in FIG. 14, for image 0(1400) and image 2 (1420) adjacent in the video content edited accordingto the FRC editing technique, the processor 210 may check a differencebetween dynamic tone metadata 1402 of image 0 (1400) and dynamic tonemetadata 1422 of image 2 (1420). For example, the difference in dynamictone metadata is a difference between syntax values defined in thedynamic tone metadata, and may include a difference between tone mappingcurves representing the tones (or brightness) of the correspondingimages.

In various embodiments, at operation 1305, the electronic device (e.g.,processor 120 or 210) may determine whether the difference in dynamictone metadata between adjacent images exceeds a second threshold. Forinstance, the second threshold is a reference value for determiningwhether pieces of dynamic tone metadata are similar, and may be setdifferently for each syntax included in the dynamic tone metadata or maybe set to correspond to the sum of differences in syntax included in thedynamic tone metadata.

In various embodiments, if the difference in dynamic tone metadatabetween adjacent images exceeds the second threshold (e.g., “yes”-branchof operation 1305), at operation 1307, the electronic device (e.g.,processor 120 or 210) may update dynamic tone metadata of at least oneadjacent image. In one embodiment, in FIG. 14, if the difference betweendynamic tone metadata 1402 of image 0 (1400) and dynamic tone metadata1422 of image 2 (1420) exceeds the second threshold, the processor 210may update the dynamic tone metadata of image 2 (1420) (1424). Forexample, the processor 210 may update the dynamic tone metadata of image2 (1420) (1424) based on dynamic tone metadata of at least one of image2 (1420), image 0 (1400) adjacent to image 2 (1420), or image 4 (1430)adjacent to image 2 (1420). For instance, the updated dynamic tonemetadata may include an average of dynamic tone metadata of referenceimages or a sum of dynamic tone metadata of reference images to which aweight is applied. For example, the dynamic tone metadata of image n+1may be updated based on the average of dynamic tone metadata ofreference images as shown in Table 1 below.

TABLE 1 Dynamic tone metadata syntax value N + 1 update Dynamic tonemetadata syntax N N + 1 value tone_mapping_flag    1    1 1 knee_point_x[0]  150  300 225 knee_point_y [0] 1000 1200 1100num_bezier_curve_anchors [0]    9    9 9 bezier_curve_anchors[0][0]  500 700 600 bezier_curve_anchors[0][1]  700  700 700bezier_curve_anchors[0][2]  800  900 850 bezier_curve_anchors[0][3]  8501000 925 bezier_curve_anchors[0][4]  850 1000 925bezier_curve_anchors[0][5]  900 1200 1050 bezier_curve_anchors[0][6] 950 1200 1075 bezier_curve_anchors[0][7]  980 1300 1140bezier_curve_anchors[0][8] 1200 1400 1300

For example, Table 1 may indicate dynamic tone metadata represented bySMPTE (society of motion picture and television engineers) or ST 2094-40syntax.

In various embodiments, if the difference in dynamic tone metadatabetween adjacent images is less than or equal to the second threshold(e.g., “no”-branch of operation 1305), at operation 1309, the electronicdevice (e.g., processor 120 or 210) may maintain dynamic tone metadataof the adjacent images. In one embodiment, when the difference indynamic tone metadata between adjacent images is less than or equal tothe second threshold, the processor 210 may determine that tone mappingsof the adjacent images are similar Hence, the processor 210 maydetermine to maintain the dynamic tone metadata of the adjacent images.

FIG. 15A is a flowchart for an electronic device to generate and editdynamic tone metadata of video content to which a video concatenationtechnique is applied according to an embodiment of the disclosure.

In the following embodiments, operations may be performed in sequencebut are not necessarily performed in sequence. For example, operationsmay be performed differently from the listed order, and at least twooperations may be performed in parallel. For instance, the electronicdevice may be the electronic device 101 in FIG. 1 or the electronicdevice 201 in FIG. 2. In the following description, at least someoperations of flowchart 1500 of FIG. 15A will be described withreference to FIG. 16.

FIG. 16 is a configuration for generating video content through a videoconcatenation technique in an electronic device according to anembodiment of the disclosure.

Referring to FIG. 15A, in various embodiments, at operation 1501, theelectronic device (e.g., processor 120 in FIG. 1 or processor 210 inFIG. 2) may generate video content including some portions of aplurality of video content by using a video concatenation technique. Inone embodiment, through video concatenation editing as shown in FIG. 16,the processor 210 may generate separate content C (1620) including atleast a portion 1602 of content A (1600) and at least a portion 1612 ofcontent B (1610).

In various embodiments, at operation 1503, the electronic device (e.g.,processor 120 or 210) may check whether an image not having dynamic tonemetadata is present among a plurality of images included in the videocontent generated based on the video concatenation technique. In oneembodiment, in FIG. 16, the processor 210 may determine that image 4(1630) extracted from content A and/or image 5 (1624) extracted fromcontent B, which are included in content C (1620) generated based on thevideo concatenation technique, do not have dynamic tone metadata.

In various embodiments, if there is at least one image not havingdynamic tone metadata in the video content generated based on the videoconcatenation technique (e.g., “yes”-branch of operation 1503), atoperation 1505, the electronic device (e.g., processor 120 or 210) maygenerate dynamic tone metadata for the at least one image not havingdynamic tone metadata in the video content generated based on the videoconcatenation technique. In one embodiment, the processor 210 maygenerate dynamic tone metadata for the at least one image based ondynamic tone metadata of another image that precedes the at least oneimage in playback time and is closest to it in the original videocontent. For example, in FIG. 16, when image 4 (1630) included incontent C (1620) does not have dynamic tone metadata, the processor 210may check content A (1600) being the original video content of image 4(1630). The processor 210 may identify image 1 (1632) closest to image 4(1630) in playback time among at least one other image (e.g., image 0and/or image 1 (1632)) that precedes image 4 (1630) in playback time andhas dynamic tone metadata in content A (1600). The processor 210 maygenerate dynamic tone metadata of image 4 (1630) belonging to content C(1620) by copying dynamic tone metadata of image 1 (1632) belonging tocontent A (1600). As another example, in FIG. 16, when image 5 (1624)included in content C (1620) does not have dynamic tone metadata, theprocessor 210 may check content B (1610) being the original videocontent of image 5 (1624). The processor 210 may identify image 3 (1642)closest to image 5 (1624) in playback time among at least one otherimage (e.g., image 1 and/or image 3 (1642)) that precedes image 5 (1624)in playback time and has dynamic tone metadata in content B (1610). Theprocessor 210 may generate dynamic tone metadata of image 5 (1624)belonging to content C (1620) by copying dynamic tone metadata of image3 (1642) belonging to content B (1610). For instance, the original videocontent may indicate video content to which an image belonging to videocontent generated based on a video concatenation technique has belongedbefore application of the video concatenation technique.

In one embodiment, the processor 210 may generate dynamic tone metadatafor at least one image based on dynamic tone metadata of another imageclosest to it in playback time in the original video content. Forexample, the processor 210 may identify image 6 (1634) closest to image4 (1630) in playback time among at least one other image (e.g., image 0,image 1 (1632) and/or image 6 (1634)) that has dynamic tone metadata incontent A (1600). The processor 210 may generate dynamic tone metadataof image 4 (1630) included in content C (1620) by copying dynamic tonemetadata of image 6 (1634) belonging to content A (1600). As anotherexample, the processor 210 may identify image 3 (1642) closest to image5 (1624) in playback time among at least one other image (e.g., image 1,image 3 (1642) and/or image 7 (1644)) that has dynamic tone metadata incontent B (1610). The processor 210 may generate dynamic tone metadataof image 5 (1624) belonging to content C (1620) by copying dynamic tonemetadata of image 3 (1642) belonging to content B (1610).

In various embodiments, if there is no image not having dynamic tonemetadata in the video content generated based on the video concatenationtechnique (e.g., “no”-branch of operation 1503) or dynamic tone metadatais generated for the image not having dynamic tone metadata (e.g.,operation 1505), at operation 1507, the electronic device (e.g.,processor 120 or 210) may check the difference in dynamic tone metadatabetween the plurality of images belonging to the video content generatedbased on the video concatenation technique. In one embodiment, in FIG.16, the processor 210 may identify the difference in dynamic tonemetadata between image 10 (1622) of content A (1600) and image 5 (1624)of content B (1610) adjacent to each other at a boundary time point incontent C (1620). For example, the difference in dynamic tone metadatais a difference between syntax values defined in the dynamic tonemetadata, and may include a difference between tone mapping curvesrepresenting the tones (or brightness) of the corresponding images.

In various embodiments, at operation 1509, the electronic device (e.g.,processor 120 or 210) may determine whether the difference in dynamictone metadata between the adjacent images exceeds the second threshold.For instance, the second threshold is a reference value for determiningwhether pieces of dynamic tone metadata are similar, and may be setdifferently for each syntax included in the dynamic tone metadata or maybe set to correspond to the sum of differences in syntax included in thedynamic tone metadata.

In various embodiments, if the difference in dynamic tone metadatabetween the adjacent images exceeds the second threshold (e.g.,“yes”-branch of operation 1509), at operation 1511, the electronicdevice (e.g., processor 120 or 210) may update dynamic tone metadata ofat least one of the adjacent images. In one embodiment, in FIG. 16, ifthe difference in dynamic tone metadata between image 10 (1622) ofcontent A (1600) and image 5 (1624) of content B (1610) exceeds thesecond threshold, the processor 210 may update the dynamic tone metadataof image 5 (1624) belonging to content C (1620). For example, theprocessor 210 may update the dynamic tone metadata of image 5 (1624)included in content C (1620) based on dynamic tone metadata of at leastone of image 5 (1624), image 10 (1622) adjacent to image 5 (1624), orimage 6 (1626) adjacent to image 5 (1624), which belongs to content C(1620). For instance, the updated dynamic tone metadata may include anaverage of dynamic tone metadata of reference images or a sum of dynamictone metadata of reference images to which a weight is applied.

In various embodiments, if the difference in dynamic tone metadatabetween the adjacent images does not exceed the second threshold (e.g.,“no”-branch of operation 1509), at operation 1513, the electronic device(e.g., processor 120 or 210) may maintain the dynamic tone metadata ofthe adjacent images.

In various embodiments, the electronic device (e.g., processor 120 or210) may generate dynamic tone metadata for at least some of the imagesthat do not have dynamic tone metadata in the video content generatedbased on video concatenation editing. For instance, the at least someimages may include an image corresponding to the start position of eachoriginal video content within the video content generated based on thevideo concatenation editing.

In one embodiment, as shown in FIG. 16, when content C (1620) generatedbased on video concatenation editing includes an image that does nothave dynamic tone metadata, the processor 210 may determine whether theimage not having dynamic tone metadata is the start image of theoriginal video content within content C (1620). For example, in FIG. 16,the start image of the original video content may include image 4 (1630)with the earliest playback time among at least a portion 1602 of contentA (1600) included in content C (1620) and/or image 5 (1624) with theearliest playback time among at least a portion 1612 of content B (1610)included in content C (1620).

In various embodiments, when the image that does not have dynamic tonemetadata in content C (1620) in FIG. 16 is the start image of theoriginal video content, the processor 210 may generate dynamic tonemetadata of the corresponding image. For example, in FIG. 16, if thereis no dynamic tone metadata for image 4 (1630) of content C (1620)extracted from content A (1600), the processor 210 may generate dynamictone metadata of image 4 (1630) included in content C (1620) based ondynamic tone metadata of another image having dynamic tone metadata incontent A (1600). For instance, the dynamic tone metadata of image 4(1630) included in content C (1620) may be generated based on dynamictone metadata of the image closest to image 4 (1630) in playback timeamong other images that precede image 4 (1630) in playback time and havedynamic tone metadata in content A (1600) (e.g., original videocontent), or based on dynamic tone metadata of the image closest toimage 4 (1630) among other images having dynamic tone metadata incontent A (1600).

In one embodiment, in FIG. 16, if the image not having dynamic tonemetadata in content C (1620) is not the start image of the originalvideo content, the processor 210 may restrict generation of dynamic tonemetadata for the corresponding image. For example, when dynamic tonemetadata of image 6 of content C (1620) of FIG. 16 extracted fromcontent A (1600) does not exist, the processor 210 may restrictgeneration of dynamic tone metadata for image 6 included in content C(1620). For instance, at the time when content C (1620) is encoded, thedynamic tone metadata of an image preceding image 6 in playback time incontent C (1620) (e.g., image 4 (1630) or image 5) may be used as (orcopied to) dynamic tone metadata of image 6.

FIG. 15B is a flowchart for an electronic device to edit dynamic tonemetadata of video content to which a video concatenation technique isapplied according to an embodiment of the disclosure.

In the following embodiments, operations may be performed in sequencebut are not necessarily performed in sequence. For example, operationsmay be performed differently from the listed order, and at least twooperations may be performed in parallel. For instance, the electronicdevice may be the electronic device 101 in FIG. 1 or the electronicdevice 201 in FIG. 2. In the following description, at least someoperations of flowchart 1530 of FIG. 15B will be described withreference to FIG. 16.

Referring to FIG. 15B, in various embodiments, at operation 1531, theelectronic device (e.g., processor 120 in FIG. 1 or processor 210 inFIG. 2) may generate separate video content by extracting some portionsof the plurality of video content according to a video concatenationtechnique. In one embodiment, as shown in FIG. 16, the processor 210 maygenerate content C (1620) by extracting at least a portion 1602 ofcontent A (1600) and at least a portion 1612 of content B (1610).

In various embodiments, at operation 1533, the electronic device (e.g.,processor 120 or 210) may check the difference in dynamic tone metadatabetween the plurality of images included in the separate video contentgenerated based on the video concatenation technique. In one embodiment,in FIG. 16, the processor 210 may identify the difference in dynamictone metadata between image 10 (1622) of content A (1600) and image 5(1624) of content B (1610) adjacent to each other at a boundary timepoint in content C (1620).

In various embodiments, at operation 1535, the electronic device (e.g.,processor 120 or 210) may determine whether the difference in dynamictone metadata between the images adjacent in playback time within theseparate video content exceeds the second threshold. For instance, thesecond threshold is a reference value for determining whether pieces ofdynamic tone metadata are similar, and may be set differently for eachsyntax included in the dynamic tone metadata or may be set to correspondto the sum of differences in syntax included in the dynamic tonemetadata.

In various embodiments, if the difference in dynamic tone metadatabetween the images adjacent in playback time exceeds the secondthreshold (e.g., “yes”-branch of operation 1535), at operation 1537, theelectronic device (e.g., processor 120 or 210) may update dynamic tonemetadata of at least one of the adjacent images. In one embodiment, inFIG. 16, if the difference in dynamic tone metadata between image 10(1622) extracted from content A (1600) and image 5 (1624) extracted fromcontent B (1610) exceeds the second threshold, the processor 210 mayupdate the dynamic tone metadata of image 5 (1624) belonging to contentC (1620). For example, the processor 210 may update the dynamic tonemetadata of image 5 (1624) based on dynamic tone metadata of at leastone of image 5 (1624), image 10 (1622) adjacent to image 5 (1624), orimage 6 (1626) adjacent to image 5 (1624), which belongs to content C(1620). For instance, the dynamic tone metadata of image 5 (1624) may beupdated based on an average of dynamic tone metadata of reference images(e.g., image 10 (1622), image 5 (1624), and/or image 6 (1626)) or a sumof dynamic tone metadata of reference images to which a weight isapplied.

In various embodiments, if the difference in dynamic tone metadatabetween the images adjacent in playback time does not exceed the secondthreshold (e.g., “no”-branch of operation 1535), at operation 1539, theelectronic device (e.g., processor 120 or 210) may maintain the dynamictone metadata of the images belonging to the video content generatedbased on the video concatenation technique.

In various embodiments, for content C (1620) generated by a videoconcatenation editing technique as shown in FIG. 16, the electronicdevice (e.g., processor 120 or 210) may set a weight for updatingdynamic tone metadata based on a sequence or group of pictures (GOP)constituting content C (1620). In one embodiment, the processor 210 mayupdate the dynamic tone metadata of image 5 (1624) extracted fromcontent B (1610) by applying a weight to dynamic tone metadata of image10 (1622) extracted from content A (1600), image 5 (1624) extracted fromcontent B (1610), and image 6 (1626) extracted from content B (1610),which are consecutive in content C (1620) in FIG. 16. For instance, arelatively large weighting coefficient may be set to the dynamic tonemetadata of image 5 (1624) and image 6 (1626) extracted from content B(1610) in comparison to the dynamic tone metadata of image 10 (1622)extracted from content A (1600).

In various embodiments, the electronic device (e.g., processor 120 or210) may continuously update dynamic tone metadata of images based onthe difference in dynamic tone metadata between images belonging to theedited video content. In one embodiment, in FIG. 16, the processor 210may continuously update dynamic tone metadata of not only the image atboundary time point (e.g., image 10 (1622) of content A (1600) or image5 (1624) of content B (1610)) but also another image extracted fromcontent B (1610) (e.g., image 6 (1626) of content B (1610)).

In various embodiments, the electronic device (e.g., processor 120 or210) may update dynamic tone metadata of an image belonging to videocontent (or edited video content) based on predefined dynamic tonemetadata. In one embodiment, as shown in FIG. 16, the processor 210 maygenerate separate content C (1620) by concatenating at least a portion1602 of video content 1600 and at least a portion 1612 of video content1610. To reduce the sense of heterogeneity at the boundary time point,the processor 210 may update the dynamic tone metadata of image 10(1622) extracted from content A (1600) and/or image 5 (1624) extractedfrom content B (1610) based on predefined dynamic tone metadata. Forexample, the processor 210 may detect an image matching image 10 (1622)among a plurality of images having dynamic tone metadata stored in thememory (e.g., memory 130 in FIG. 1) and/or the server (e.g., server 108in FIG. 1). For instance, the dynamic tone metadata of image 10 (1622)may be updated based on the dynamic tone metadata of the image matchingimage 10 (1622). As another example, the processor 210 may detect animage matching image 5 (1624) among the plurality of images havingdynamic tone metadata stored in the memory (e.g., memory 130 in FIG. 1)and/or the server (e.g., server 108 in FIG. 1). For instance, thedynamic tone metadata of image 5 (1624) may be updated based on thedynamic tone metadata of the image matching image 5 (1624). Forinstance, at least one image having dynamic tone metadata stored in thememory or server may include at least one image generated (or obtained)by the electronic device 201 at a previous point in time and/or at leastone image generated by an expert.

FIG. 17 is a flowchart for an electronic device to edit dynamic tonemetadata of video content to which clip editing is applied according toan embodiment of the disclosure.

In the following embodiments, operations may be performed in sequencebut are not necessarily performed in sequence. For example, operationsmay be performed differently from the listed order, and at least twooperations may be performed in parallel. For instance, the electronicdevice may be the electronic device 101 in FIG. 1 or the electronicdevice 201 in FIG. 2. In the following description, at least someoperations of flowchart 1700 of FIG. 17 will be described with referenceto FIG. 18.

FIG. 18 is a configuration for generating video content through clipediting in an electronic device according to an embodiment of thedisclosure.

Referring to FIG. 17, in various embodiments, at operation 1701, theelectronic device (e.g., processor 120 in FIG. 1 or processor 210 inFIG. 2) may generate fourth video content by extracting at least aportion of first video content based on a clip editing technique. In oneembodiment, as shown in FIG. 18, the processor 210 may generate contentD (1810) including at least a portion 1802 of content A (1800).

In various embodiments, at operation 1703, the electronic device (e.g.,processor 120 or 210) may check whether there is an image not havingdynamic tone metadata among a plurality of images belonging to thefourth video content generated based on the clip editing technique. Inone embodiment, in FIG. 18, the processor 210 may determine that image 2(1820) does not have dynamic tone metadata in content D (1810).

In various embodiments, if there is at least one image not havingdynamic tone metadata (e.g., “yes”-branch of operation 1703), atoperation 1705, the electronic device (e.g., processor 120 or 210) maygenerate dynamic tone metadata for the at least one image not havingdynamic tone metadata. In one embodiment, the processor 210 may generatedynamic tone metadata for the at least one image based on dynamic tonemetadata of another image that precedes it in playback time and isclosest to it in the original video content. For example, in FIG. 18,the processor 210 may identify image 0 (1822) closest to image 2 (1820)in playback time among at least one other image that precedes image 2(1820) in playback time and has dynamic tone metadata in content A(1800) being the original video content of content D (1810) includingimage 2 (1820). The processor 210 may generate dynamic tone metadata ofimage 2 (1820) by copying dynamic tone metadata of image 0 (1822)belonging to content A (1800). In one embodiment, the processor 210 maygenerate dynamic tone metadata for the at least one image based ondynamic tone metadata of another image closest to it in playback time inthe original video content. For example, in FIG. 18, the processor 210may identify image 3 (1824) closest to image 2 (1820) in playback timeamong at least one other image having dynamic tone metadata in content A(1800) (e.g., image 0 (1822) and/or image 3 (1824)). The processor 210may generate dynamic tone metadata of image 2 (1820) by copying dynamictone metadata of image 3 (1824) belonging to content A (1800).

In various embodiments, if there is no image not having dynamic tonemetadata (e.g., “no”-branch of operation 1703), the electronic device(e.g., processor 120 or 210) may store the fourth video contentgenerated based on the clip editing technique. For instance, the fourthvideo content may be stored in the memory (e.g., memory 130 in FIG. 1)of the electronic device 201 or the server (e.g., server 108 in FIG. 1).

In various embodiments, the electronic device (e.g., processor 120 or210) may generate dynamic tone metadata for at least some of thoseimages not having dynamic tone metadata in the video content generatedbased on clip editing. For instance, the at least some images mayinclude an image corresponding to the start position of the originalvideo content within the video content generated based on clip editing.

In one embodiment, when content D (1810) generated based on clip editingincludes an image not having dynamic tone metadata as shown in FIG. 18,the processor 210 may determine whether the image not having dynamictone metadata is the start image of the original video content (e.g.,content A (1800)). For instance, the start image of the original videocontent may include image 2 (1820) with the earliest playback time amongat least a portion 1802 of content A (1800) included in content D (1810)in FIG. 18.

In one embodiment, if the image not having dynamic tone metadata incontent D (1810) in FIG. 18 is the start image of the original videocontent, the processor 210 may generate dynamic tone metadata of thecorresponding image. For example, if there is no dynamic tone metadatafor image 2 (1820) included in content D (1810) in FIG. 18, theprocessor 210 may generate dynamic tone metadata of image 2 (1820) basedon dynamic tone metadata of another image having dynamic tone metadatain content A (1800). For instance, the dynamic tone metadata of image 2(1820) may be generated based on dynamic tone metadata of an imageclosest to image 2 (1820) among other images that precede image 2 (1820)in playback time in content A (1800) and have dynamic tone metadata, ordynamic tone metadata of an image closest to image 2 (1820) among otherimages that have dynamic tone metadata in content A (1800).

In one embodiment, if the image not having dynamic tone metadata incontent D (1810) in FIG. 18 is not the start image of the original videocontent, the processor 210 may restrict generation of dynamic tonemetadata for the corresponding image. For example, when dynamic tonemetadata of image 4 of content D (1810) in FIG. 18 does not exist, theprocessor 210 may restrict generation of dynamic tone metadata for image4. For instance, at the time when content D (1810) is encoded, thedynamic tone metadata of an image preceding image 4 in playback time incontent D (1810) (e.g., image 2 (1820) or image 3) may be used as (orcopied to) dynamic tone metadata of image 4.

In various embodiments, the electronic device 201 may generate dynamictone metadata of an image not having dynamic tone metadata based on aplurality of other images in video content generated by applying a videoconcatenation technique or clip editing technique. In one embodiment,when there are multiple other images having dynamic tone metadata in theoriginal image content, the processor 210 may generate dynamic tonemetadata of an image not having dynamic tone metadata based on anaverage of dynamic tone metadata of the other images. In one embodiment,the processor 210 may generate dynamic tone metadata of an image nothaving dynamic tone metadata by applying a weight corresponding to thedifference in playback time from the image not having dynamic tonemetadata to the other images having dynamic tone metadata in theoriginal video content.

FIG. 19 is a flowchart for an electronic device to update dynamic tonemetadata of an edited image according to an embodiment of thedisclosure.

In the following embodiments, operations may be performed in sequencebut are not necessarily performed in sequence. For example, operationsmay be performed differently from the listed order, and at least twooperations may be performed in parallel. For instance, the electronicdevice may be the electronic device 101 in FIG. 1 or the electronicdevice 201 in FIG. 2.

Referring to FIG. 19, in various embodiments, at operation 1901 offlowchart 1900, the electronic device (e.g., processor 120 in FIG. 1 orprocessor 210 in FIG. 2) may edit at least one image belonging to videocontent based on a video editing technique (e.g., scaling, objectinsertion, or filtering). In one embodiment, the processor 210 mayenlarge or reduce the size of the whole or partial region of at leastone image included in the video content based on a scaling editingtechnique. In one embodiment, the processor 210 may insert graphic ortext data into at least a partial region of at least one image includedin the video content based on an object insertion editing technique. Inone embodiment, the processor 210 may filter the whole or partial regionof at least one image included in the video content based on a filteringediting technique.

In various embodiments, at operation 1903, the electronic device (e.g.,processor 120 or 210) may detect an image matching the edited imageamong those images having dynamic tone metadata stored in the memory(e.g., memory 130 in FIG. 1) or a server (e.g., server 108 in FIG. 1).In one embodiment, the processor 210 may select an image having thesmallest difference from the edited image, among those images stored inthe memory or the server, as an image matching the edited image. Forinstance, the difference from the edited image may indicate a differencein pixel value or feature point between the edited image and an imagestored in the memory or server. For example, the images stored in thememory or the server may include at least one image generated by theelectronic device 201 using a camera (e.g., camera module 180 in FIG. 1)and/or at least one image generated by an expert.

In various embodiments, at operation 1905, the electronic device (e.g.,processor 120 or 210) may identify the dynamic tone metadata of theimage matching the edited image. For instance, the image matching theedited image may indicate an image having a difference from the editedimage less than or equal to the first threshold among the images storedin the memory or server.

In various embodiments, at operation 1907, the electronic device (e.g.,processor 120 or 210) may generate dynamic tone metadata of the editedimage based on dynamic tone metadata of the image matching the editedimage. In one embodiment, the processor 210 may update the dynamic tonemetadata of the edited image by copying dynamic tone metadata of theimage matching the edited image. For instance, the processor 210 mayupdate dynamic tone metadata of all edited images belonging to the videocontent.

In various embodiments, the electronic device (e.g., processor 120 or210) may update the dynamic tone metadata of the image edited based on avideo editing technique in a manner corresponding to the video editingtechnique. In one embodiment, when the size of at least one imagebelonging to the video content is enlarged based on a scaling technique,the processor 210 may update the dynamic tone metadata of the at leastone enlarged image so as to correspond to the enlarged image.

FIGS. 20A to 20C illustrate encoding schemes of video content in anelectronic device according to various embodiments of the disclosure.

For instance, the electronic device may be the electronic device 101 inFIG. 1 or the electronic device 201 in FIG. 2.

Referring to FIG. 20A, the bit stream of images having dynamic tonemetadata in video content may be configured. For instance, each ofindividual images 2000 to 2030 may be configured in a form in whichdynamic tone metadata M0, M1, M2 or M3 is positioned in front of imagedata P0, P1, P2 or P3. For example, in the case of HDR 10+, dynamic tonemetadata may be included in supplemental enhancement information (SEI).For instance, a picture order count (POC) of each of the images 2000 to2030 in video content indicates the display order, and may be includedin the slice header.

In various embodiments, the electronic device (e.g., processor 120 or210) may insert at least one additional image into the video contentbased on an FRC editing technique.

Referring to FIG. 20B, the processor 210 may generate image 0.5 (2040),image 1.5 (2042) and image 2.5 (2044) and insert them into the videocontent. For example, each additional image 2040, 2042 or 2044 may beconfigured in a form in which dynamic tone metadata M0.5, M1.5 or M2.5is located at the front end of image data P0.5, P1.5 or P2.5. In oneembodiment, as the additional images 2040, 2042 and 2044 are insertedinto the video content with respect to the time axis, the POC of theimages 2000 to 2030 set as shown in FIG. 20A may be changed. Forexample, the POCs of the additional images 2040, 2042 and 2044 may beset respectively to 1, 3 and 5 based on the display order, and the POCsof the original images 2010 to 2030 may be changed respectively to 2, 4and 6 (2012 to 2032).

In various embodiments, the electronic device (e.g., processor 120 or210) may encode the additional images 2040, 2042 and 2044 inserted intothe video content based on the FRC editing technique by adding them tothe bit stream of the original images.

Referring to FIG. 20A, among the original images 2000 to 2030 of thevideo content, image 0 (2000) may be encoded as an intra picture (e.g.,I picture), and image 1 (2010), image 2 (2020) and image 3 (2030) may beencoded as a P picture. For instance, image 3 (2030) may be encoded withreference to image 2 (2020), image 2 (2020) may be encoded withreference to image 1 (2010), and image 1 (2010) may be encoded withreference to image 0 (2000). In one embodiment, the additional images2040, 2042 and 2044 inserted into the video content may be added to andencoded in the bit stream of the original images 2000, 2010, 2020 and2030. For example, the additional images 2040, 2042 and 2044 may beencoded as an intra picture. As another example, the additional images2040, 2042 and 2044 may be encoded as a P picture with reference toimage 0 (2000). As another example, the additional images 2040, 2042 and2044 may be encoded as a P picture with reference to an image precedingin POC.

Referring to FIG. 20C, the electronic device (e.g., processor 120 or210) may encode the additional images 2040, 2042 and 2044 inserted intothe video content based on an FRC editing technique independently of theoriginal images. In one embodiment, among the additional images 2060,2062 and 2064 of the video content, as shown in FIG. 20C, image 1 (2060)may be encoded as an intra picture (e.g., I picture), and image 3 (2062)and image 5 (2064) may be encoded as a P picture. For instance, image 5(2064) may be encoded with reference to image 3 (2062), and image 3(2062) may be encoded with reference to image 1 (2060).

According to various embodiments of the disclosure, when editing videocontent including dynamic tone metadata (e.g., frame rate upconversion), the electronic device may generate dynamic tone metadata ofa new image based on dynamic tone metadata of at least one referenceimage, so that dynamic tone mapping can be applied while shortening thegeneration time of dynamic tone metadata of the new image.

According to various embodiments, when editing video content includingdynamic tone metadata (e.g., frame rate down conversion or videoconcatenation), the electronic device may edit dynamic tone metadata ofat least one image based on a difference in dynamic tone metadatabetween adjacent images to thereby reduce a sense of heterogeneity forimages with respect to the tone in the video content.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. An electronic device comprising: a display; andat least one processor operatively connected to the display, wherein theat least one processor is configured to: generate a third image to beinserted between a first image and a second image continuous with thefirst image among a plurality of images belonging to video content,based on at least one of the first image or the second image, generatedynamic tone metadata of the third image based on dynamic tone metadataof the first image and dynamic tone metadata of the second image, andupdate the video content by adding the third image and the dynamic tonemetadata of the third image.
 2. The electronic device of claim 1,wherein the at least one processor is further configured to generate thedynamic tone metadata of the third image based on an average of thedynamic tone metadata of the first image and the dynamic tone metadataof the second image.
 3. The electronic device of claim 1, wherein the atleast one processor is further configured to: identify a weightcorresponding to the first image and the second image based on at leastone of a reference ratio of the first image and the second imagereferenced to generate the third image, a time interval between thefirst image, the second image, and the third image, a sequence includingthe first image, the second image, and the third image, or informationon a group of pictures (GOP) including the first image, the secondimage, and the third image, and generate the dynamic tone metadata ofthe third image by applying the weight to the dynamic tone metadata ofthe first image and the dynamic tone metadata of the second image. 4.The electronic device of claim 1, further comprising a memory to store aplurality of images having dynamic tone metadata, and wherein the atleast one processor is further configured to: select an imagecorresponding to the third image from among the plurality of imagesstored in the memory or a plurality of images stored in anotherelectronic device, and set the dynamic tone metadata of the third imageto dynamic tone metadata of the selected image corresponding to thethird image.
 5. The electronic device of claim 1, wherein the at leastone processor is further configured to: generate the third image basedon the second image in case that a difference between the first imageand the second image satisfies a specified condition, and generate thethird image based on the first image and the second image in case thatthe difference between the first image and the second image does notsatisfy the specified condition.
 6. The electronic device of claim 5,wherein in case that the third image is generated based on the secondimage, the at least one processor is further configured to set thedynamic tone metadata of the third image to the dynamic tone metadata ofthe second image.
 7. An electronic device comprising: a display; and atleast one processor operatively connected to the display, wherein the atleast one processor is configured to: generate first video content byextracting at least one image from one or more video contents, identify,in case that there is a first image not having dynamic tone metadataamong at least one image included in the first video content, originalvideo content including the first image among the one or more videocontents, identify at least one image having dynamic tone metadata amonga plurality of images included in the original video content, select animage that satisfies a specified condition in playback time from amongthe identified at least one image having dynamic tone metadata, andgenerate dynamic tone metadata of the first image based on dynamic tonemetadata of the selected image.
 8. The electronic device of claim 7,wherein the at least one processor is further configured to: identify atleast one image preceding the first image in playback time among one ormore images of the original video content having dynamic tone metadata,and select an image closest to the first image in playback time amongthe at least one image preceding the first image in playback time. 9.The electronic device of claim 7, wherein the at least one processor isfurther configured to select an image closest to the first image inplayback time among at least one image of the original video contenthaving dynamic tone metadata.
 10. The electronic device of claim 7,wherein the at least one processor is further configured to generate thedynamic tone metadata of the first image by copying dynamic tonemetadata of the selected image.
 11. The electronic device of claim 7,wherein the at least one processor is further configured to generate thefirst video content by extracting at least one image from second videocontent.
 12. The electronic device of claim 7, wherein the at least oneprocessor is further configured to generate the first video content byconnecting at least one image included in second video content and atleast one image included in third video content.
 13. The electronicdevice of claim 12, wherein the at least one processor is furtherconfigured to: compare dynamic tone metadata of a second image being alast of at least one image of the second video content included in thefirst video content with dynamic tone metadata of a third image being afirst of at least one image of the third video content included in thefirst video content, wherein the third image is continuous with thesecond image within the first video content, and update the dynamic tonemetadata of the third image based on the dynamic tone metadata of thesecond image in case that a difference in dynamic tone metadata betweenthe second image and the third image satisfies a specified condition.14. The electronic device of claim 13, wherein the at least oneprocessor is further configured to update the dynamic tone metadata ofthe third image based on the dynamic tone metadata of at least one ofthe second image, the third image, or a fourth image continuous with thethird image in case that the difference in dynamic tone metadata betweenthe second image and the third image satisfies the specified condition.15. The electronic device of claim 14, wherein the at least oneprocessor is further configured to update the dynamic tone metadata ofthe third image based on an average of dynamic tone metadata of at leasttwo of the second image, the third image, or the fourth image.
 16. Theelectronic device of claim 14, wherein the at least one processor isfurther configured to: identify a weight corresponding to the secondimage, the third image, and the fourth image based on at least one of asequence including the second image, the third image and the fourthimage, or information on a group of pictures (GOP) including the secondimage, the third image and the fourth image, and update the dynamic tonemetadata of the third image by applying the weight to dynamic tonemetadata of at least one of the second image, the third image, or thefourth image.
 17. The electronic device of claim 13, wherein the atleast one processor is further configured to maintain the dynamic tonemetadata of the third image in case that the difference in dynamic tonemetadata between the second image and the third image does not satisfythe specified condition.
 18. An electronic device comprising: a memoryto store a plurality of images having dynamic tone metadata; a display;and at least one processor operatively connected to the memory and thedisplay, wherein the at least one processor is configured to: update afirst image among a plurality of images belonging to video content basedon video editing, select a second image corresponding to the updatedfirst image from among the plurality of images stored in the memory or aplurality of images stored in another electronic device, set dynamictone metadata of the updated first image by updating dynamic tonemetadata of the first image based on dynamic tone metadata of the secondimage corresponding to the updated first image.
 19. The electronicdevice of claim 18, wherein the at least one processor is furtherconfigured to set the dynamic tone metadata of the updated first imageby copying the dynamic tone metadata of the second image correspondingto the updated first image.
 20. The electronic device of claim 18,wherein the video editing includes at least one of scaling, objectinsertion, or filtering.